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The Basics of Differential Evolution 


> population-based optimization algorithm 

> Introduced by Storn and Price in 1997. 

> Developed to optimize real parameter , ; real valued functions. 

> General problem formulation is: 

For an objective function / ; X czR D —> R 

where the feasible region 0 , the minimization problem is to find 

x* e X such that f(x*) < f(x) V x & X 

Where: 


f(X*) ^ - GO 


The Basics of Differential Evolution 


> DE is a parallel direct search method which utilizes NP 
/^-dimensional parameter vectors. 

> Suppose we want to optimize a function with D dimensional real 
parameters R D . 

> We must select the size of the population NP (TVPmust be > 4), 

X IG = [ X l,i,G’ X 2J,G ’ * * * X DJ,gJ 

i - 1, 2, . . . ,NP and j = 1, 2, ... ,D 

where x iG is a parameter vector in a population for each generation 
and G is the generation number. 

> TVPdoes not change during the minimization process. 


Differential Evolution (DE) 


> DE is an Evolutionary Algorithm. 

> DE is constructed from initialization and a cycle of stages of 
mutation, recombination (or crossover), and selection. 



Figure 1: Basic Stages of DE 






Initialization 




> all parameter vectors in a population are randomly initialized . 

> Define upper and lower bounds for each parameter: 

xi < X i < X^ 

J ~ J,i,l ~ J 

> Randomly select the initial parameter values uniformly on the 
intervals 

[ Xj L , A'/'] 

> Suggestion to chose random values between high bound and low 
bound 


x ui = 4 + rand \4 - 4) 





Mutation 


Initialization 


Mutation 




Selection 



> Mutation , recombination and selection will be run for each of the NP 
parameter vectors of a population. 

> For a given parameter vector x iG (called target vector) randomly 
select three vectors x rl G , x r2 G and x r3G such that the indexes l rl, r2 
and r3 are distinct integers {1, 2 , . . . ,NP}. 

> Add the weighted difference of the two vectors x r2G , x r3G 
to the base vector x rl G . 

Vi,G+l =x rlG +F.( x r2G — x r3G ) 





Mutation 


Initialization 


Mutation 




Selection 



> The mutation factor F is a constant from [0 , 2] which controls the 
amplification of the differential variation ( x r2G ~ x r3G ). 

> v iG+1 is called the mutant vector or donorxz ctor. 





• NP parameter vectors from generation G 
0 Mutated parameter vector v iG+1 


X2 



Figure 2: An example of a two-dimensional cost function showing the 
process for generating the mutant vector v iG+1 by three different 
vectors x rlG , x r2G and x r3G . 


Crossover 


Initialization 


Mutation 


Crossover 


Selection 




> Crossover incorporates successful solutions from the previous 
generation. 

> The trial vector u iG+1 is developed from the elements of the target 
vector x i G and the elements of the mutant vector v i G+1 


u 


j ,i, G+l 


v J i G+] if ( randb(j ) < CR ) or j = Irand 
Xj . G if (randb(j) > CR ) and j ^ Irand 








> randb(j) is the j th evaluation of a uniform random number generator 
with out come ^[0,1]. 

> CR is called “ the crossover rate ” and it is a constant [0 , 1] which 
has to be determined by the user. 

> Irand is a random integer from [1, 2, ...,D] which ensures that the trial 
vector u iG+ , gets at least one parameter from v iG+1 
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Figure 3: Illustration of the crossover process for £>= 6 parameters. 





Selection 

> After the crossover process, some or all the components of the trial 
vectors may be lie outside the search domain. 

> So, to be sure that all components of the generating trial vectors 
over all iterations are within the predefined boundary constraints, 
one of the following three equations is used: 


Crossover 







Crossover 


Initialization 



or, 


xf 4- randji. (x jXG - xf) , if (u UG+1 > xf) 
j.i.c+i ) z, + rand ( X _ vf) , if (u, XG+1 < xf ) 


( 1 ) 


u j,i,G+l 


2xf - u jXG+ 1 , if (U;,;,G+1 > *;) 
- U ;,i,G+l . if (W;,i,G+l < xf) 


U' 


( 2 ) 


or, 


u 


j,i,G+ 1 


(xf+Xj,i, G )/ 2 , if (u UtG+1 > xf) 
(xf+xjxc)/ 2 , if (%i, G +i < xf) 


( 3 ) 





Selection 


Initialization 




Crossover 


Selection 




> The target vector x i G is compared with the trial vector u iG+1 and the 
one with the lowest cost function value is chosen to the next 
generation 


x i,G+l 


u i,G + 1 > ^ (/( u i,G + 1) — 

Xi G , "otherwise" 


> Mutation, recombination and selection continue until some stopping 
criterion is reached 





Meaning of the name? 
DE/rand/l/bin 


> DE: Differential Evolution 

> rand: Base vector for mutation is chosen randomly 

> 1: one difference vector is used to construct the donor 

> bin: crossover is binomial 


Other Variants of DE Schemes 


The mutant vector v iG+1 is generated according to one of the following 
equations: 

> V i,G+l = x rl,C +F ( X r2,G ~ X r3,G ) (!) 

DE/rand/l/bin 

> n G+1 = x best, G +F -( X rl,G~ X r2,G) ( 2 ) 

DE/best/l/bin 

> n G+1 = X i,G +F ( X best, G~ X 1 ,g) +F ( x rl, G ~ X r2 , g) (3) 


DE/target-to-best/l/bin 


Other Variants of DE Schemes 


> V i G+l = X best, G + R ( X rl,G ~ X r2, G ) + F ( X r3, G ~ X r4 , g) 

DE/best/2/bin 

> n G + 1 = X rl,G + P-( X r2, G ~ X r3 , g) + F ( X r4, G ~ X r5 , g) 


(4) 

(5) 


DE/rand/2/bin 


DE Control Parameters 


> The parameters that control the performance of DE are three : 

(1) The population size NP 

(2) The mutation factor F 

(3) The crossover rate CR 

> These parameters should be chosen (or tuned) carefully to avoid the 
state of stagnation (or premature convergence) for the DE algorithm. 


DE Control Parameters 


> NP affects the ability to search the parameter space. 

• TVPmust be > 4 (why?). 

• Small values of NP result in few numbers of mutant vectors that may 
cause insufficient exploration (premature convergence). 

• On the other hand, large values of NP result in many numbers of 
mutant vectors that may cause excessive exploration (slow 
convergence) and increase the number of computations. 

• In [4] 1 VP = 30 for all small dimension values D < 30 and NP -D for 
large dimension values D> 30. 


DE Control Parameters 


> The mutation factor F is relevant to the convergence speed as it is 
responsible for the step size that interferes in the formation of the mutant 
vector. 

• Small values of F will lead to premature convergence 

• F> 1 will try to take large steps, leading to slow convergence. 

• A good initial choice of F is 0.5 and the effective range usually lies in 
[0.4, 1] as Storn and Price suggested in [1]. 


DE Control Parameters 


> The crossover rate CR controls the number of changes in parameters of a 
population member . 

• A small value of CR ('strong' crossover e.g. 0 or 0.1) leads to most 
changes occurring along one dimension or a small subset of 
dimensions, and this is useful for separable functions . 

• Large values of CR (near 1) lead to most components being chosen 
from the mutant vector and this is suitable for non-separable functions . 

• In [5], CR lies in [0, 0.2] when the function is separable and lies 
in [0.9, 1] when it is non-separable. 


References 

[1] Storn, R. and Price, K. (1997), ‘Differential Evolution - A Simple and 
Efficient Heuristic for Global Optimization over Continuous Spaces’, 
Journal of Global Optimization, 11, pp. 341-359. 

[2] Price, K., Storn, R., Lampinen, A., “Differential Evolution: A Practical 
Approach to Global Optimization”, Springer-Verlag Berlin Heidelberg 
2005. 

[3] Uday K. Chakraborty, ” Advances in Differential Evolution”, Springer- 
Verlag Berlin Heidelberg 2008 . 

[4] Noman, N., and Iba, H., "Accelerating differential evolution using an 
adaptive local search", IEEE Transactions on Evolutionary Computation, 
vol. 12, no. 1, pp. 107-125, Feb. 2008. 

[5] Ronkkonen J., Kukkonen, S., and Price, K., "Real-parameter optimization 
with differential evolution", in Proc. IEEE CEC, vol. 1, pp. 506-513, 
2005. 


